(this document is updated as the weeks proceed)
- Intro video to Alex Holcombe, find it in Canvas
Week 7: Stats and studies: Correlation and causation
- Correlation and causation intro
- Learning objectives:
- Understand and apply three causal models to explaining correlations
- Know the term “spurious correlation”
- Where does data come from?
- Understanding correlation more deeply
- Three causal models
- X causes Y, Y causes X, a third variable causes both
- Causal model can be at any level of detail
- Explanatory, Outcome, and Nuisance variables
- Recall IV and DV. Bruce slide:

- Nuisance variable
- Is not different, on average, for the different levels of the variable we are interested in
- Confound variables
- Different for the different levels of the variable we are interested in.
- Bruce slide:

- Can you distinguish between confounds and nuisance variables?
- Random assignment
- Bruce slide:

- Randomisation and control group
- Bruce slide

- Time as a third variable
- Relationship between pirates and high average temperature is confounded by time.
- Time: Post hoc ergo propter hoc (After this therefore because of this) fallacy
Week 8
- Dichotomous correlation
- dichotomous variables and correlation
- Contingency tables
- Later you will see the link to arguments and logic
- 4 kinds of control
- Statistical adjustment - controlling for confounds
- A controlled variable - matching
- Causal phrases verus correlational phrases, see “Distinguish correlational and causal..” video on Canvas

Week 9
- Hypothesis testing review and extension
- Review of hypothesis testing (recall tutorial week 5 and Bruce’s 18 March lecture)
- Bruce slide:

- False positives, false negatives, true positives, true negatives
- Hypothesis testing and medical testing
- Sensitivity and specificity
- Why does the news have lots of false positives?
- Excess of statistical comparisons
- More reasons given later in the class, after logic and arguments
Arguments and logic , Week 9-
We’re going to go back and forth between bare-bones examples and arguments from the wild, giving you more and more tools to deal with the real-world ones.
Week 9: Fallacies.
- Informal fallacies
- Post hoc ergo propter hoc, again
- Video combining this fallacy with deductive and inductive logic. Good way to bring previous bits of the class together!
- Ad hominem:
Week 10 -
- Wason card selection task, as an argument/syllogism
- Mindset again: Biased evaluation - confirmation or “my-side” bias
- It stops search for possibilities (Manley chapter 2)
- Francis Bacon was concerned, so he invented science
- Mercier & Sperber
- Info sources that tend to use opposing experts
- Not invented here
- Overconfidence
- 53 slides above for this week
Week 11
- Why does the news have lots of false positives?
- Excess of statistical comparisons (already talked about this)
- Publication bias
- P-hacking
- Preregistration
- Expectancy effects
- Experimenter
- Participant
- Addressing expectancy effects

- Experimentation effect
Base rates, probabilities, and correlations
- How probabilities are presented
- Probabilities versus frequencies
- When you hear a probability, think of 100 or 1000 cases
- Correlations seemingly implied by rates
- Which bits of the contingency table are given by a particular statement?
Is there a correlation between being male and liking avocados?
Suppose we learn that most males like avocados and also that most people who like avocadoes are male. Can we conclude that liking avocadoes is correlated with being male? It’s tempting to think the answer is “yes.” Understanding why that’s the wrong answer is crucial to having a full understanding of correlation.
For two dichomotous variables like male/female and liking avocados versus not, a positive correlation would mean that a higher proportion of males like avocado than females.
We know two things * Most males like avocado * Most people who like avocado are male
To establish a correlation, what we need to know is whether males like avocados at a higher rate than females do. But that simply does not follow from the fact that most males like avocados and most who like avocado are male. Look at this example:
Number of males versus females who like and don’t like avocados
| No |
40 |
38 |
| Yes |
60 |
58 |
Most males in this example
What proportion of males like avocados? What proportion of females like avocados
- Most people in the world like avocados. So we should expect most males to like avocados even if the same proportion of males and females like avocados.
Most people in the world are male (by a small amount). So we should expect most avocado eaters to be male even if males and females like avocados at the same rate.
Putting these two facts together still doesn’t give us a correlation, because they could both be true even if males and females own cell phones at the same rate. - Make a 2 x 2 table. For base rate of each - being male and owning a cellphone.
Most northern Hemisphere residents have above world average income. Most people with above world average income are in the northern hemisphere.
Most N. Hemisphere countries have more than 300 COVID-19 deaths. Most countries with more than 300 COVID-19 deaths are in the Northern Hemisphere. Is there a correlation between being in the N. Hemisphere and having more than 300 COVID-19 deaths? Maybe not, because most countries are in the N. Hemisphere anyway, so the second statement doesn’t tell us much. We need more to know whether the proportion of northern hemisphere countries with >300 COVID-19 deaths is greater than the proportion of southern hemisphere countries.
- Another kind of mistake is simply that we fail to think proportionally. For example, suppose we've only observed John when it's cold and we notice that he has worn a hat 70% of the time. Can we conclude that there is a correlation in our observations between his wearing a hat and cold temperatures? Of course not! What if he wears a hat 70% of the time regardless of the temperature? In that case, there is no special correlation between his hat wearing and the cold: he just loves wearing hats.
If we are told, "Most of the time when it's cold, John wears a hat," it's easy to forget that this is not enough to establish a correlation. To infer a correlation, we have to assume that John does not wear a hat most of the time on other days too . Maybe this is a safe assumption to make, but maybe not. The point is that if we just ignore it, we are neglecting the base rate, a mistake we encountered in the previous chapter.
- Probability We aren’t good at understanding probabilities. Gerd Gigerenzer argues we are poor at understanding probabilities, but better at frequencies.
Overconfidence takes over and we tend to think we can beat the odds “statistics happen to other people.” In risky financial markets this can get people into a lot of trouble. E.g., most people lose their money in futures markets But the spectacular profits that can be gained draw in people who believe they will be the ones to win. Awareness of our biases and heuristics could improve our thinking. * Risk * Relative versus absolute risk Relative versus absolute risk Relatively risky https://twitter.com/justsaysrisks
- Who to believe
- Telling fact from fiction in the news
- Ensure your mouth and throat are always moist. Stomach acid can kill coronavirus. It’s just the flu. China created COVID-19 as a biological weapon. At least 60 per cent of the population needs to be exposed to build up “necessary resistance” to the virus. From this
- Only about 30% of Australians checked with other sources of info before sharing something
- OK but there are other signals of credibility
- And background knowledge
- Argument from authority
- Bias: Incentive, and circumstantial ad hominem
- Track record
- Postdiction versus prediction “ Heyman notes that Gottman doesn’t predict divorce at all. He postdicts it. He gets 100 (or however many) couples, sees how many divorced, and then finds a set of factors that explain what happened.”
- Experts
- Expertise: area of expertise
- Success bias p.10 of Mercier - Makes sense in small-scale societies
- Multiple independent experts
- Wikipedia (pp.263-5 of Lyons & Ward)
- Fake experts
- “Fact or Fake news? Evaluating Sources” OLEO1645 by Michelle Harrison
- “Evaluate the credibility of claims and sources” from Think Critically by Facione
- Video is a favorite of fake news
- Connecting correlation interpretation to base rate neglect. Do it before section on media - say “and both base rate neglect and media stuff takes us back to correlations because we’re getting more data in some of the cells of the 2 x 2 than others”
Learning from mistakes
- This stormtrooper has realized that he made a big mistake. That’s good because then he can learn from the mistake.
The knew-it-all-along effect
Related to not wanting to admit that one is wrong.
This guy
- In retrospect thinks he knew it all along
- The observers are skeptical. They are probably right that he didn’t predict this.
- But in retrospect, people think that they did predict something.
- He didn’t predict it, so the best thing to do is realize his error, like the stromtrooper, so he can potentially learn from it
- If one doesn’t know one made an error, one won’t learn much from the error. And wrong theories of the world never get fixed!
“I knew he would win!”
Prior to the 2012 election, average person said likelihood of Obama winning was 59%.
After the election, the average person (different set of people) said 68% (p < .001).
Hindsight bias
Ulkumen, Tannnenbaum, & Fox
Pundits carry on thinking all their political theories are correct.
Kahan has shown that evoking curiousity can help. Use it on yourself, too - I’m curious why you feel that way.
- Conversations
- Mindset: humility
- Why people don’t change their mind much
- The naive view
- Challenging a person’s ideas with facts will cause them to change their position
- The best thing to do is point to exactly where they are wrong
- Why people don’t change their mind
- Don’t want to admit they were wrong
- Confirmation bias
- Knew-it-all-along effect (hindsight bias)
- Ideally people would learn from their mistakes. But often humans show the knew-it-all-along effect.
- ADD Media bias - section 5.3 of Manley
- Incentive to make research more meaningful than it is
- Just says in mice
- Causation in headline when it’s an observational study
- Look back at Distinguishing correlational and causal statements video for an example
- Example

- ADD Research media bias - publication bias (Bruce slide #130) and file drawer problem - section 5.3 of Manley
- The internet
- No gatekeeper or vetting (p. 256 of Lyons & Ward) +1
- Unvetted sites can look just as slick as vetted sites, not true in the old days
- Google Pagerank (crowdsourced vetting) +1
- extremism (rare people can find and reinforce each other) +1
- Anonymity
- Spurious corroboration (sites copying each other)
Having an argument can extremize both sides
Taking a specific criticism as a general one
Don’t leave your shoes in the middle of the floor
You always leave your shoes in the middle of the floor
Avoiding entering argument mode
Daniel Pink on How to Persuade Others with the Right Questions
This guy
I’m not one of those people who went into psychology because they wanted to deal with feelings
What I’m into is evidence, reasons and logic. But I’ve learned I can only have those conversations with certain people if I deal with their feelings.
Survivorship * Bruce’s slide #132 * Famous people are usually very good, but also very lucky * “Failure to look for what is missing is a common shortcoming” https://youarenotsosmart.com/2013/05/23/survivorship-bias/ covers Wald, * Heuristic: Chesterton’s fence, as selection bias?